A virtual reality (VR) system represents a three-dimensional computer graphic (3D CG) created by a computer to the user, thus allowing the user to experience a virtual space as if it were real. In recent years, a technique that presents information which is not available in the real world by compositing a 3D CG onto a video of the real world has been developed, and such technique is called an AR (Augmented Reality) system or MR (Mixed Reality) system.
An AR system can superimpose a 3D CG onto a real object. For example, a game described in Japanese Patent Laid-Open No. 2000-353248 (reference 1) (corresponding US Publication Number: 2002-072416) superimposes a 3D CG of a sword, other weapon, or the like onto an interactive operation input device held by the user, thus allowing the user to freely manipulate a virtual object (in this case, a sword or other weapon).
In the apparatus described in reference 1, when the user holds the interactive operation input device at a specific posture, the contents of a 3D CG to be superimposed on the interactive operation input device change. More specifically, if the user holds the device like a sword, a 3D CG of the sword is superimposed on the interactive operation input device; if he or she holds the device like a shield, a 3D CG of the shield is superimposed on the interactive operation input device.
In this way, in order to switch a virtual object to be superimposed on a real object, the user must make a specific gesture or use an input device such as a mouse or the like. For this reason, when the user wants to change a 3D CG while holding a real object and observing the 3D CG of a virtual object superimposed on that object, he or she must make a gesture or operate the input device, and must pause an ongoing operation.